Breast cancer screening using mammography has demonstrated that early detection is one obvious way to increase survival. However, due to the limitations of the observers, not all potentially detectable cancers are reported. Studies (see C. J. Baines, D. V. McFarlane, and A. B. Miller, “The role of the reference radiologist: estimates of interobserver agreement and potential delay in cancer detection in the national cancer screening study,” Invest Radiology, 25, pp. 971-976, 1990; R. E. Bird, T. W. Wallace, and B. C. Yankaskas, “Analysis of cancers missed at screening mammography,” Radiology, 184, pp. 613-617, 1992; J. A. Harvey, L. L. Fajardo, and C. A. Innis, “Previous mammograms in patients with impalpable breast carcinoma: retrospective vs blinded interpretation,” AJR, 161, pp. 1167-1172, 1993) showed that 20-30% of cancers known to be visible on mammograms were missed by radiologists. The misses have been classified into two categories, namely search miss and interpretation miss (see H. L. Hundel, C. F. Nodine, and D. Carmody, “Visual scanning, pattern recognition and decision making in pulmonary nodule detection,” Invest Radiology, vol. 13, pp. 175-181, 1978; C. J. Savage, A. G. Gale, E. F. Pawley, and A. R. M. Wilson, “To err is human, to compute divine,” Proc. Second Int. Workshop on Digital Mammography [Elsevier, Amsterdam], A. G. Gale et al Eds, York UK, pp. 405-414, 1994). A search miss occurs when the radiologist simply does not see the abnormality. An interpretation miss occurs when the radiologist sees something but interprets it as something else. This invention provides an image visualization method to assist radiologists in overcoming these two types of misses in breast cancer diagnosis.
Studies also show that a second reading can increase by 5-15% the detection rate of breast cancer (see F. L. Thurfjell, K. A. Lernevall, and A. A. S. Taube, “Benefit of independent double reading in a population-based mammography screening program,” Radiology, 191, pp. 241-244, 1994; R. E. Bird, “Professional quality assurance for mammographic screening programs,” Radiology, 177, pp. 587-597, 1990). However, a second reading is expensive and time consuming. It reduces the efficient use of a radiologist's time as well as causing delays in patient through-put. Less than 2% of breast clinics do a second reading. Computer-aided diagnosis (CADx), computer-aided detection (CAD), and image analysis technique have shown potential for replacing a second reading.
CADx methods usually estimate the likelihood of the malignancy of a tumor that is detected by CAD. Current methods of CAD produce markers that indicate abnormalities. These marks are typically circles, arrows, or some form of annotation. The drawback of these methods is that numerous false positives distract radiologists from the subtle signs of disease not detected by CAD. The study conducted by Sittek et al (see H. Sittek and M. F. Reiser, “Initial clinical experience with CAD in mammography,” Computer-Aided Diagnosis in Medical Imaging, K. Doi et al Eds, pp. 185-191, 1999) showed the false positive rate to be 94% in the only clinically available CAD-System ImageChecker (R2 Technology, USA). Image processing techniques can be used to improve the visualization of mammographic images. In one embodiment of the present invention, the contrast of an area of particular interest in a digital mammographic image, for example, the dense breast parenchymal pattern, is enhanced to help radiologists interpret the images. Another advantage of contrast enhancement is to overcome the dynamic range and modulation transfer function limitations of the output display media, such as radiographic film and monitor display.
More than 70% of breast cancers develop in the parenchymal zone. Cancers do not arise in the fat tissues because fat cells never divide. The breast parenchymal pattern contains the functional glandular elements (also called ductal elements, which are the milk channels) and stroma (connective and supporting tissues). The interpretation of a mammogram requires that radiologists visualize the 2 dimensional mammogram as the projection of a 3D anatomic object and to search ductal networks to detect cancers. In order to improve the ability of a radiologist to visually detect cancer on a digital mammogram, the appearance of breast parenchyma pattern must be enhanced relative to the fatty-tissue surroundings.
Contrast enhancement methods can be divided into two categories. The first category is called the single scale approach, where the image is processed in the original image domain, e.g., a simple look-up-table is used to transform the image. The second category is called the multi-scale approach, where the image is decomposed into multiple resolution scales and processing is performed in the multi-scale domain before the image is reconstructed back to the original image intensity domain.
The most common single scale methods make use of contrast stretch windowing as described in the work of Aylward et al (see S. R. Aylward, B. M. Hemminger, and E. D. Pisano, “Mixture modeling for digital mammogram display and analysis,” Proc. Fourth Int. Workshop on Digital Mammography, [Kluwer, Bonston], N. Karssemeijer et al Eds, Nijmegen, Netherland, pp. 305-312, 1998). Because windowing is applied to one of the segmented image regions, such as uncompressed dense fat and muscle, at the expense of contrast reduction in other segments, a consequence of single scale methods is that the contrast in the area of dense breast tissue is increased at the cost of decreased contrast in the over-penetrated (darker) areas of the image, e.g., near the skin line. Mutihac et al (see R. Mutihac, A. A. Colavita, A. Cicuttin and A. E. Cerdeira, “Maximum entropy improvement of X-ray digital mammograms,” Proc. Fourth Int. Workshop on Digital Mammography [Kluwer, Bonston], N. Karssemeijer et al Eds, Nijmegen, Netherland, pp. 329-336, 1998) describe a contrast enhancement method by maximizing entropy of the digital mammograms. This procedure requires prior knowledge to be stated as a set of constraints on the input image, for example, noise is independent of pixel value. For digital mammograms, this assumption is often invalid because of a strong correlation among neighboring pixels. These methods also require reliable estimates of standard deviations that are based on theoretical or experimental data. Morrow et al (see W. M. Morrow, R. B. Paranjape, R. M. Rangayyan, and J. E. Leo Desautels, “Region-based contrast enhancement of mammograms,” IEEE Transactions on Medical Imaging, vol. 11, no. 3, pp. 392-405, 1992) describe another contrast enhancement method based on region growing. The region growing is initiated at the area of interest in test mammogram images that were identified with the aid of an experienced radiologist. A tolerance parameter k is selected such that if a pixel value is between (1−k)f to (1+k)f, then the pixel belongs to the region, where f is the gray value of the starting pixel. A new contrast is then reassigned based on the property of the region. Because the initial seed is selected manually, the practical utility of this method is reduced.
Among multi-scale approaches, un-sharp masking (USM) is a special case since the image is decomposed into two scales. The image is first passed through a low-pass filter to obtain a low resolution (low frequency) image. The high frequency image is obtained by subtracting the low frequency image from the original image. The high frequency components are then amplified and added back to the low frequency components to form a new image that has enhanced image detail. Many variations of USM have been developed. The method of Tahoces et al (see P. G. Tahoces, J. Correa, M. Souto, C. Gonzalez, and L. Gomez, “Enhancement of chest and breast radiography by automatic spatial filtering,” IEEE Transactions on Medical Imaging, vol. 10, no. 3, pp. 330-335, 1991) combines two different un-sharp masking sizes (7×7 and 25×25) to achieve enhancement of high and median frequencies. Chang et al (see D. C. Chang and W. R. Wu, “Image contrast enhancement based on a histogram transformation of local standard deviation,” IEEE Transactions on Medical Imaging, vol. 17, no. 4, pp. 518-531, 1998) and Narendra et al (see P. M. Narendra and R. C. Fitch, “Real-time adaptive contrast enhancement,” IEEE Transactions on Pattern Ana. Machine Intell., vol. PAMI-3, pp. 655-661, 1981) describe a method of adaptively adjusting the gain of high frequency components based on the local standard deviation. Ji et al (see T.-L. Ji, M. K. Sundareshan, and H. Roehrig, “Adaptive image contrast enhancement based on human visual properties,” IEEE Transactions on Medical Imaging, vol. 13, no. 4, pp. 575-586, 1994) describe how to adjust high frequency components by using human visual properties. These three methods are variations of the well-known un-sharp masking technique applied to digital mammography. All three methods cause dark and bright banding artifacts around high contrast edges, known as edge-banding artifacts. The edge-banding artifact is objectionable in areas where there are rapid changes in the image, for example, at the sharp boundaries between the skin line and the air-background in a mammogram.
To deal with the edge-banding problem, methods have been developed that are based on decomposing an image into multiple resolutions and using a predetermined nonlinear function to compress the high amplitude of the high frequency component in each resolution. Vuylsteke et al (see U.S. Pat. No. 5,467,404, “Method and apparatus for contrast enhancement,” Nov. 14, 1995, P. P. Vuylsteke and E. P. Schoeters), Clark et al (see U.S. Pat. No. 5,982,917, “Computer-assisted method and apparatus for displaying X-ray images,” Nov. 9, 1999, L. P. Clarke, W. Qian, and L. Li), Laine et al (see A. F. Laine, S. Schuler, J. Fan, and W. Huda, “Mammographic feature enhancement by multi-scale analysis,” IEEE Transactions on Medical Imaging, vol. 13, no. 4, pp. 725-740, 1994), and Strickland et al (see R. N. Strickland, L. J. Baig, W. J. Dallas, and E. A. Krupinski, “Wavelet-based image enhancement as an instrument for viewing CAD data,” Proc. Third Int. Workshop on Digital Mammography [Elsevier, Amsterdam], K. Doi et al Eds, Chicago, Ill., pp. 441-446, 1996) used the combination of linear and nonlinear compression functions for the high frequency components in each resolution. Takeo et al (see U.S. Pat. No. 6,014,474, “Image processing method and apparatus,” Jan. 11, 2000, H. Takeo, M. Yamada and N. Nakajima) used a morphology operator as a nonlinear compression function for the high frequency components. In the above-mentioned methods, if the high frequency amplitude is large, then it is attenuated. If the high frequency amplitude is small, it is passed unchanged or even amplified. All these methods assume that whenever the signal amplitude gets large, regardless of the cause, the amplitude should be reduced such that overshoot and undershoot artifacts near sharp edges are properly suppressed. The general purpose of contrast enhancement is to facilitate useful image information to be visually extracted without the addition of artifacts. To attain optimal contrast enhancement of an image for visual interpretation, two problems are to be addressed. The first is to identify specifically which image details should be enhanced. The second problem is to determine the degree of enhancement. The methods described in the prior art fail to address these two problems adequately for digital mammography. First, the amplitude information at each resolution is inadequate to determine whether a large amplitude is caused by a high contrast edge (e.g., from muscle to fat) or an area of interest (e.g., from ducts to fat). Failure to make the distinction between useful edges and abundant edges will also cause the suppression of fine details associated with high amplitudes, resulting in a loss of important image detail. Second, a predetermined nonlinear amplitude compression function is not sufficient to adaptively enhance the amount of image detail for each resolution at different spatial locations.
There is an image modality, called xerography, which produces edge-enhanced images that have a look similar to the mountain-view presentation. Applying the xerographic reproduction process to the breast began in 1956. The major differences between conventional radiography and Xeroradiography are: (1) the latent image is formed by photoconduction on a selenium surface plate and developed by a dry process; (2) a permanent image is recorded on opaque paper; (3) edge enhancement is often greater. The edges of the denser area in the breast are usually more pronounced than in fine-grain film mammography. Xeroradiography was also somewhat more successful in imaging the breast with an implant. This is not only because the wide latitude of the imaging process permitted simultaneous display of the full range of native breast tissues and the more opaque implant itself, but also because the standard lateral projection image routinely depicted all portions of the implant, including its posterior margins (see E. A. Sickles, “Breast imaging: from 1965 to the present,” Radiology, 215, pp. 1-16, 2000). However, Xerography has been discontinued in breast imaging for about 25 years. The major disadvantages of xerography are: (1) more radiation by a large factor; (2) messy (due to toner process); (3) more expensive.